External systems generate growing numbers of events, such as social media posts and notifications. Furthermore, legacy systems can generate large volumes of events for processing by applications. Customers are increasingly looking to build or leverage enterprise applications that are capable of receiving and processing high volumes of events in short periods of time from these external or legacy systems. These systems can be used in a variety of applications ranging from health care to automotive repair. The applications can facilitate a range of business operations, from marketing and manufacturing to distribution to technical support. For example, an application can implement data-processing workflows to support processing transactional data ranging from customer service requests received by retail and banking enterprises, to routing and resolution of health care claims by insurance enterprises.
Computers can be networked into a cluster that includes two or more nodes capable of exchanging events or messages. The cluster can include one or more producer nodes and one or more consumer nodes. The producer nodes can transmit, or publish, events and the consumer nodes can receive, or subscribe to, events. Traditional techniques for exchanging and processing messages include a publish-subscribe model. In a publish-subscribe model, publishers or producers do not transmit messages direct to an individual subscriber or consumer. Instead, publishers transmit a message broadly to multiple subscribers at once. The subscribers generically register interest in certain messages and thereby receive only messages of interest.
However, the publish-subscribe model suffers from reduced throughput because a system uses valuable capacity in the network to provide a given message to multiple consumer nodes, in case more than one consumer node registers interest in the message. Furthermore, the publish-subscribe architecture generally uses an intermediate middleware node sometimes referred to as a message broker or message bus for message delivery, using additional resources in the cluster. A still further downside of traditional message processing systems is consumer nodes generally receive messages in any order, instead of preserving the order in which the producer nodes read the messages from a message stream for faster message routing and processing.
An object of this invention is to provide improved systems and methods for digital data processing. A more particular object is to provide improved systems and methods for event processing.
A further object is to provide such improved systems and methods as facilitate deployment of enterprise applications in a networked cluster of producer nodes and consumer nodes.
Yet a still further object is to provide such improved systems and methods as better utilize computing and networking resources upon addition or removal of producer nodes and consumer nodes to or from the cluster.